A Dual Role for Prediction Error in Associative Learning

Confronted with a rich sensory environment, the brain must learn statistical regularities across sensory domains to construct causal models of the world. Here, we used functional magnetic resonance imaging and dynamic causal modeling (DCM) to furnish neurophysiological evidence that statistical associations are learnt, even when task-irrelevant. Subjects performed an audio-visual target-detection task while being exposed to distractor stimuli. Unknown to them, auditory distractors predicted the presence or absence of subsequent visual distractors. We modeled incidental learning of these associations using a Rescorla-Wagner (RW) model. Activity in primary visual cortex and putamen reflected learning-dependent surprise: these areas responded progressively more to unpredicted, and progressively less to predicted visual stimuli. Critically, this prediction-error response was observed even when the absence of a visual stimulus was surprising. We investigated the underlying mechanism by embedding the RW model into a DCM to show that auditory to visual connectivity changed significantly over time as a function of prediction error. Thus, consistent with predictive coding models of perception, associative learning is mediated by prediction-error dependent changes in connectivity. These results posit a dual role for prediction-error in encoding surprise and driving associative plasticit

A dual role for prediction error in associative learning

Confronted with a rich sensory environment, the brain must learn statistical regularities across sensory domains to construct causal models of the world. Here, we used functional magnetic resonance imaging and dynamic causal modeling (DCM) to furnish neurophysiological evidence that statistical associations are learnt, even when task-irrelevant. Subjects performed an audio-visual target-detection task while being exposed to distractor stimuli. Unknown to them, auditory distractors predicted the presence or absence of subsequent visual distractors. We modeled incidental learning of these associations using a Rescorla--Wagner (RW) model. Activity in primary visual cortex and putamen reflected learning-dependent surprise: these areas responded progressively more to unpredicted, and progressively less to predicted visual stimuli. Critically, this prediction-error response was observed even when the absence of a visual stimulus was surprising. We investigated the underlying mechanism by embedding the RW model into a DCM to show that auditory to visual connectivity changed significantly over time as a function of prediction error. Thus, consistent with predictive coding models of perception, associative learning is mediated by prediction-error dependent changes in connectivity. These results posit a dual role for prediction-error in encoding surprise and driving associative plasticity

Challenging the negative learning bias hypothesis of depression: Reversal learning in a naturalistic psychiatric sample

Background Classic theories posit that depression is driven by a negative learning bias. Most studies supporting this proposition used small and selected samples, excluding patients with comorbidities. However, comorbidity between psychiatric disorders occurs in up to 70% of the population. Therefore, the generalizability of the negative bias hypothesis to a naturalistic psychiatric sample as well as the specificity of the bias to depression, remain unclear. In the present study, we tested the negative learning bias hypothesis in a large naturalistic sample of psychiatric patients, including depression, anxiety, addiction, attention-deficit/hyperactivity disorder, and/or autism. First, we assessed whether the negative bias hypothesis of depression generalized to a heterogeneous (and hence more naturalistic) depression sample compared with controls. Second, we assessed whether negative bias extends to other psychiatric disorders. Third, we adopted a dimensional approach, by using symptom severity as a way to assess associations across the sample. Methods We administered a probabilistic reversal learning task to 217 patients and 81 healthy controls. According to the negative bias hypothesis, participants with depression should exhibit enhanced learning and flexibility based on punishment v. reward. We combined analyses of traditional measures with more sensitive computational modeling. Results In contrast to previous findings, this sample of depressed patients with psychiatric comorbidities did not show a negative learning bias. Conclusions These results speak against the generalizability of the negative learning bias hypothesis to depressed patients with comorbidities. This study highlights the importance of investigating unselected samples of psychiatric patients, which represent the vast majority of the psychiatric population

Adaptive and aberrant reward prediction signals in the human brain

Theories of the positive symptoms of schizophrenia hypothesize a role for aberrant reinforcement signaling driven by dysregulated dopamine transmission. Recently, we provided evidence of aberrant reward learning in symptomatic, but not asymptomatic patients with schizophrenia, using a novel paradigm, the Salience Attribution Test (SAT). The SAT is a probabilistic reward learning game that employs cues that vary across task-relevant and task-irrelevant dimensions; it provides behavioral indices of adaptive and aberrant reward learning. As an initial step prior to future clinical studies, here we used functional magnetic resonance imaging to examine the neural basis of adaptive and aberrant reward learning during the SAT in healthy volunteers. As expected, cues associated with high relative to low reward probabilities elicited robust hemodynamic responses in a network of structures previously implicated in motivational salience; the midbrain, in the vicinity of the ventral tegmental area, and regions targeted by its dopaminergic projections, i.e. medial dorsal thalamus, ventral striatum and prefrontal cortex (PFC). Responses in the medial dorsal thalamus and polar PFC were strongly correlated with the degree of adaptive reward learning across participants. Finally, and most importantly, differential dorsolateral PFC and middle temporal gyrus (MTG) responses to cues with identical reward probabilities were very strongly correlated with the degree of aberrant reward learning. Participants who showed greater aberrant learning exhibited greater dorsolateral PFC responses, and reduced MTG responses, to cues erroneously inferred to be less strongly associated with reward. The results are discussed in terms of their implications for different theories of associative learning

Nonlinear dynamic causal models for fMRI

Models of effective connectivity characterize the influence that neuronal populations exert over each other. Additionally, some approaches, for example Dynamic Causal Modelling (DCM) and variants of Structural Equation Modelling, describe how effective connectivity is modulated by experimental manipulations. Mathematically, both are based on bilinear equations, where the bilinear term models the effect of experimental manipulations on neuronal interactions. The bilinear framework, however, precludes an important aspect of neuronal interactions that has been established with invasive electrophysiological recording studies; i.e., how the connection between two neuronal units is enabled or gated by activity in other units. These gating processes are critical for controlling the gain of neuronal populations and are mediated through interactions between synaptic inputs (e.g. by means of voltage-sensitive ion channels). They represent a key mechanism for various neurobiological processes, including top-down (e.g. attentional) modulation, learning and neuromodulation. This paper presents a nonlinear extension of DCM that models such processes (to second order) at the neuronal population level. In this way, the modulation of network interactions can be assigned to an explicit neuronal population. We present simulations and empirical results that demonstrate the validity and usefulness of this model. Analyses of synthetic data showed that nonlinear and bilinear mechanisms can be distinguished by our extended DCM. When applying the model to empirical fMRI data from a blocked attention to motion paradigm, we found that attention-induced increases in V5 responses could be best explained as a gating of the V1→V5 connection by activity in posterior parietal cortex. Furthermore, we analysed fMRI data from an event-related binocular rivalry paradigm and found that interactions amongst percept-selective visual areas were modulated by activity in the middle frontal gyrus. In both practical examples, Bayesian model selection favoured the nonlinear models over corresponding bilinear ones